Evaluating Digital Libraries: A Longitudinal and Multifaceted View

published or submitted for publicatio

A comparison of Boolean-based retrieval to the WAIS system for retrieval of aeronautical information

An evaluation of an information retrieval system using a Boolean-based retrieval engine and inverted file architecture and WAIS, which uses a vector-based engine, was conducted. Four research questions in aeronautical engineering were used to retrieve sets of citations from the NASA Aerospace Database which was mounted on a WAIS server and available through Dialog File 108 which served as the Boolean-based system (BBS). High recall and high precision searches were done in the BBS and terse and verbose queries were used in the WAIS condition. Precision values for the WAIS searches were consistently above the precision values for high recall BBS searches and consistently below the precision values for high precision BBS searches. Terse WAIS queries gave somewhat better precision performance than verbose WAIS queries. In every case, a small number of relevant documents retrieved by one system were not retrieved by the other, indicating the incomplete nature of the results from either retrieval system. Relevant documents in the WAIS searches were found to be randomly distributed in the retrieved sets rather than distributed by ranks. Advantages and limitations of both types of systems are discussed

Category-Based and Association-Based Map Displays by Human Subjects

Visual displays are important for information retrieval. A good visual display should not only help the viewer to see and sense the information, but also abstract and present the semantic structure of the underlying information to facilitate searching and browsing. Depending on the type of structures shown on the display, visual displays for information retrieval can be classified as hierarchical displays, network displays, scatter displays, and map displays (LiD, 1993)

From Information Retrieval to Information Interaction

Abstract. This paper argues that a new paradigm for information retrieval has evolved that incorporates human attention and mental effort and takes advantage of new types of information objects and relationships that have emerged in the WWW environment. One aspect of this new model is attention to highly interactive user interfaces that engage people directly and actively in information seeking. Two examples of these kinds of interfaces are described

User studies informing E-table interfaces

In this paper, we describe a series of user studies that were used to advance understanding of how people use electronic tables (E-Tables) and inform the development of a web-based statistical table browser for use by non-specialists. Interviews and focus groups with providers, intermediaries, and non-specialist end users; transaction log analysis; and email content analysis were used to develop a user-task taxonomy for government statistical data. These studies were the basis for a prototype web-based interface for browsing statistical tables. Two usability studies with 23 subjects and two eye-tracking studies with 21 subjects were conducted with this interface and paper, PDF, and spreadsheet interfaces. The results of the needs assessment, prototype development, and user studies provide a foundation for understanding E-Tables in general and guiding continued design of interfaces for E-Tables

A Study on Video Browsing Strategies

Due to the unique characteristics of video, traditional surrogates and control/browsing mechanisms that facilitate text-based information retrieval may not work sufficiently for video. In this paper, a video browsing interface prototype with key frames and fast play-back mechanisms was built and tested. Subjects performed two kinds of browsing-related tasks: object identification and video comprehension under different display speeds (1 fps, 4 fps, 8 fps, 12 fps and 16 fps). It was found that browsing the key frames between 8 to 12 fps could potentially define a functional limit in object identificationaccuracy. There was no significant performance difference found across display speeds tested. The results also showed that lower speeds were required for object identification than for video comprehension. How user performance was affected by individual characteristics such as age, gender, academic background and TV- or movie-watching habits, was investigated, but no significant difference was found due to the limit of sample size and other constraints. (Also cross-referenced as UMIACS-TR-97-40) (Also cross-referenced as CLIS-TR-97-06

Interfaces and Tools for the Library of Congress National Digital Library

This paper describes a collaborative effort to explore user needs in a digital library, develop interface prototypes for a digital library, and suggest and prototype tools for digital librarians and users at the Library of Congress (LC). Interfaces were guided by an assessment of user needs and aimed to maximize interaction with primary resources and support both browsing and analytical search strategies. Tools to aid users and librarians in overviewing collections, previewing objects, and gatherin g results were created and serve as the beginnings of a digital librarian toolkit. The design process and results are described and suggestions for future work are offered. (Also cross-referenced as UMIACS-TR-98-09